Current recognition algorithm of pneumonia image faces two problems. First, the extracted features can not fit the pneumonia image well because the transfer learning model used by the pneumonia feature extractor has large image difference between the source dataset and the pneumonia dataset. Second, the softmax classifier used by the algorithm can not well process high-dimensional features, and there is still room for improvement in recognition accuracy. Aiming at these two problems, a pneumonia image recognition algorithm based on Deep Convolution Neural Network (DCNN) was proposed. Firstly, the GoogLeNet Inception V3 network model trained by ImageNet dataset was used to extract the features. Then, a feature fusion layer was added and random forest classifier was used to classify and forecast. Experiments were implemented on Chest X-Ray Images pneumonia standard dataset. The experimental results show that the recognition accuracy, sensitivity and specificity of the proposed model reach 96.77%, 97.56% and 94.26% respectively. The proposed model is 1.26 percentage points and 1.46 percentage points higher than the classic GoogLeNet Inception V3+Data Augmentation (GIV+DA) algorithm in the index of recognition accuracy and sensitivity, and is close to the optimal result of GIV+DA in the index of specificity.